Trialog: How Peer Collaboration Helps Remediate Errors in an ITS

Many intelligent tutoring systems (ITSs) offer feedback and guidance through structured dialogs with their students, which often take the form of a sequence of hints. However, it is often difficult to replicate the complexity and responsiveness of human conversation with current natural language understanding and production technologies. Although ITSs reveal enough information to continue solving a problem, the conversations are not very engaging. To enhance engagement, the current study manipulated tutorial dialog by transforming them into a trialog by adding another student. Our intention was to advance the help offered by the system by putting students in a position to help each other, as well as make sense of the help offered by the ITS. The present paper attempts to show that conversations, either with the system or with a peer, are important design considerations when building an effective ITS. Interactive Tutoring Why should we expect that learning from interactive tutoring leads to stronger learning gains than non-interactive instruction? Intuitively, it makes sense that students should form a deeper understanding when they are asked a series of questions by the tutor in which they are expected to reply in natural language. But what does the empirical data suggest? The answer is mixed (VanLehn et al., 2007). There is positive evidence to suggest that interactive tutoring leads to stronger learning gains than less interactive learning situations (such as reading). For instance, Evens and Michael developed an interactive tutoring environment in which medical students learn about the baroreceptor reflex, which is a part of the circulatory system that maintains a constant blood pressure across a variety of postures and conditions. Students using their system demonstrated larger learning gains than a control group that was asked to read a text written to match the content of the tutor (Rovick & Michael, 1992). There is also convincing evidence to the contrary, that non-interactive learning situations are equally effective as interactive tutoring. For instance, Craig et al. (2006) contrasted an interactive tutor, called AutoTutor, with a modified version of the same system that only presented a Copyright © 2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. didactic lecture. The lecture was non-interactive in the sense that students only watched the lecture. They were not required to respond to the system. The students demonstrated equal learning gains in both conditions. This result was surprising, especially because the two systems were closely matched for the content that was presented to the students. Because there is convincing evidence, both for and against interactive tutoring, it appears that several variables are interacting, in complex ways, to produce the observed learning gains. One possible avenue for further exploration is to look at the dialogs and hints produced by the tutoring systems themselves. Making sense of automatically generated hints When an authority speaks, novices tend to listen. However, what if the authority does not make sense from the student’s perspective? What are novices to do then? Consider the following hint provided by Andes, an intelligent tutoring system (ITS) for physics: “You should finish entering all of the useful given quantities in the problem. Why don't you work on entering the given value of the magnitude of the electric field at the region due to an unspecified agent.” If you are unfamiliar with electric fields, you may not be aware that an electric field is a vector quantity, which is expressed as both a magnitude and a direction. Moreover, you may not be familiar with the way in which this particular tutoring system handles vectors and their magnitude representations. The bottom line is that this hint may not be helpful to the uninitiated. Not surprisingly, novices sometimes find themselves in this unfortunate position while using an ITS. For whatever reason, the help emanating from an ITS might not be all that helpful. When this happens, the student soon learns either: a.) to drill down to the terminal (or “bottom-out”) hint, or b.) randomly input slightly different entries until the system marks it as correct. Because the help system is unable to sufficiently guide students around their current impasse, these “help abusers” can miss potentially important learning opportunities. What can be done to rectify this situation? One approach is to observe students’ interactions with the help system and modify them to promote positive help-seeking behaviors. For instance, Baker, et al. (2006) designed an ITS to detect when students abuse the help system. When Proceedings of the Twenty-First International FLAIRS Conference (2008)